On the Validity of Repeated Assessments in the UMAT, a High-Stakes Admissions Test

Autor/inn/en	Andrich, David; Styles, Irene; Mercer, Annette; Puddey, Ian B.
Titel	On the Validity of Repeated Assessments in the UMAT, a High-Stakes Admissions Test
Quelle	In: Advances in Health Sciences Education, 22 (2017) 5, S.1245-1262 (18 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Mercer, Annette)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1382-4996
DOI	10.1007/s10459-017-9761-6
Schlagwörter	High Stakes Tests; College Entrance Examinations; Undergraduate Study; Selection Criteria; Medical Education; Health Sciences; Item Response Theory; Probability; Goodness of Fit; Test Validity; Foreign Countries; Australia; New Zealand + Suchen Sie Ihr Suchwort? Aufnahmeprüfung; Grundstudium; Selection criterion; Auslesekriterium; Medizinische Ausbildung; Item-Response-Theorie; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Testvalidität; Ausland; Australien; Neuseeland
Abstract	The possibility that the validity of assessment is compromised by repeated sittings of highly competitive and high profile selection tests has been documented and is of concern to stake-holders. An illustrative example is the Undergraduate Medicine and Health Sciences Admission Test (UMAT) used by some medical and dental courses in Australia and New Zealand. The proficiencies of all applicants who sat the UMAT from one to four sittings between 2006 and 2012 were estimated on the same metric using the probabilistic Rasch model. A fit index characterising each profile's degree of conformity to the model was also calculated. Confirming expectations, mean proficiencies increased with repeated sittings on all three UMAT scales with the greatest difference (which was nevertheless relatively small) between the first two sittings. The fit index showed that the increases in proficiency estimates arose from additional easier items being answered correctly on repeated sittings rather than additional more difficult ones, suggesting that improvements are not on the substantive construct of the variable of assessment but in skills in answering the questions. Although strategies for dealing with the increase in proficiency estimates on repeated sittings could be canvassed, these results suggest that the validity of results on repeated sittings was not compromised. Accordingly, it might be concluded that although particular individuals might improve substantially between sittings, any validity is not likely to be compromised with the possibility that for some applicants, the second sitting might be the most valid. (As Provided).
Anmerkungen	Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: service-ny@springer.com; Web site: http://www.springerlink.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01